Method and apparatus for photograph finding

ABSTRACT

Digital image data including discrete photographic images of a variety of different subjects, times, and so forth, are collected and analyzed to identify specific features in the photographs. In an embodiment of the invention, distinctive markers are distributed to aid in the identification of particular subject matter. Facial recognition may also be employed. The digital image data is maintained in a database and quarried in response to search requests. The search requests include criteria specifying any feature category or other identifying information, such as date, time, and location that each photograph was taken, associated with each photograph. Candidate images are provided for review by requesters, who may select desired images for purchase or downloading.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.13/090,026, filed Apr. 19, 2011, which is a continuation of Ser. No.12/874,929, filed Sep. 2, 2010 (now U.S. Pat. No. 7,929,810), which is acontinuation of Ser. No. 12/325,589, filed Dec. 1, 2008 (now U.S. Pat.No. 7,844,141), which is a continuation of Ser. No. 11/056,699, filedFeb. 10, 2005 (now U.S. Pat. No. 7,460,737), which claims priority toU.S. Provisional Appl. No. 60/544,570, filed Feb. 12, 2004; thedisclosures of each of the above-referenced applications areincorporated by reference herein in their entireties.

BACKGROUND OF THE INVENTION Field of the Invention

Photography has transformed how people conceive of the world.Photographs allow people to see all sorts of things that are actuallymany miles away and/or years preceding. Photography lets people capturemoments in time and preserve them for years to come.

Often people at a public place notice that a stranger has taken aphotograph of which they would love to have a copy, Alternatively, aftergoing somewhere, a person may bemoan the fact that he did not have aphotograph of the event (in the present context, photograph alsoincludes video, audio, or other representation).

A need exists, therefore, to provide a method and apparatus foridentifying and connecting people with photographs they want. Inaddition, there is a need to provide a method and apparatus forcharacterizing errant photographs stored on computer databases thatmakes use of a variety of attributes to reliably characterizephotographs in such a way as to reduce the amount of manual reviewnecessary to identify and connect people with the photographs they want.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus that matchespeople with photographs in which they accidentally (or purposely) appearor with photographs of events they have attended.

Specifically, in one embodiment, a web site is created with a databasebackend. The database is seeded with information provided by (1) thephotographer; (2) recovering metadata from the photograph; (3) readingdevices such as a Global Positioning System (GPS) device; (4)referencing the account data of the photographer (i.e., account number,photographer's zip code or area code, etc.); (5) analyzing thephotograph (i.e., computer recognizes eye color, optical characterrecognizes any text found in the photograph, recognizes the number ofpersons, the gender of persons, the hair color, the time of day byoptical character recognizing any clocks in the photograph or analyzingthe lighting conditions, the weather, etc.); (6) photograph qualityinformation; and/or (7) any other information.

A user looking for a photograph would visit the web site and search forcertain criteria. The user is then provided with a gallery of thumbnailsthat match the criteria. When the user identifies a photograph he wantsto own, he can then download the full quality version, or orderprint(s). In a preferred implementation, the user is charged some amountof money that is split between the site owner and the photographer.Alternatively, the user may be charged in some other way, such as byviewing advertisements or by exchanging credits for downloads or by someother payment or a combination thereof.

A more complete understanding of the present invention will be affordedto those skilled in the art, as well as a realization of additionaladvantages and objects thereof, by a consideration of the followingdetailed description of the preferred embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow diagram showing exemplary steps of a method accordingto the invention.

FIG. 2 is a diagram showing an exemplary distinctive marker forphotographic data.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention provides a method and apparatus that matchespeople with photographs in which they accidentally (or purposely) appearor with photographs of events they have attended.

FIG. 1 illustrates exemplary steps of a method 100 according to theinvention. At optional step 102, distinctive markers may be distributedto persons desiring to contribute photographic images to a database. Themarkers may comprise, for example, distinctive bins, badges, or stickersfor placing on objects to be photographed. The markers should bedesigned so as to be easily recognized using automatic recognitionalgorithms, but should not be too conspicuous.

At step 104, image data is collected from a variety of sources. It maybe desirable to accept material from as many sources as possible, toincrease the number of images available for browsing. Optionally, imagesmay be accepted from qualified sources only.

At step 104, source information regarding each photograph is collected.For example, at step 108, the image provider may be asked forinformation, such as the time and date of the photograph, the subjectmatter, location, photographer, etc. Image metadata may also be read toobtain such information. Other ways of associating images to identifyinginformation may include, for example, reading devices such as a GlobalPositioning System (GPS) device attached to a camera or other imagecapture device, or by referencing account data of the image contributor(e.g., account number, photographer's zip code or area code, etc.).

At step 110, image data is analyzed to identify any characteristics thatmay be of interest to users. Such characteristics may include, forexample, eye color, words and sentences, a number or gender of persons,the hair color, time of day, lighting conditions, and so forth. Forfurther example, at step 112, a facial recognition program as known inthe art may be used to analyze any faces appearing in the photos at asufficiently high resolution. At step 114, the images may be analyzedfor the presence of any known markers. And at step 116, other featuresand qualities of the image may be classified, for example, whether it istaken indoors or outdoors, whether it contains people, dogs, cats, orother animals, whether it contains automobiles, airplanes, or otherobjects, and so forth. At step 118, selected feature information andother source information is associated with each image and provided toany suitable relational database.

At step 120, requests specifying search criteria for photographic imagesare received. For example, a user looking for a photograph may visit aweb site hosted by the system and fill out a search form specifyingsearch criteria of interest. The criteria may include specific subjectmatter, times, dates, and locations. For example, “Disneyland ANDMatterhorn AND blue eye AND child AND Jan. 1, 2004 AND morning” wouldsearch for a photograph or photographs taken at Disneyland's Matterhornwith a child who has blue eyes on the morning of Jan. 1, 2004.

At step 122, the image database is queried as known in the art, toidentify images that at least partially match the search criteria. Suchimages may be presented, at step 124, to the user. For example, the usermay be provided with a gallery of “thumbnail” (reduced-size) imagesgenerated from images that match the criteria. When the user identifiesa photograph he wants to own, he can then download the full qualityversion, or order print(s). In a preferred implementation, the user ischarged some amount of money that is split between the site owner andthe photographer. Alternatively, the user may be charged in some otherways such as by viewing advertisements or by exchanging credits fordownloads or by some other payment or a combination thereof. The pricecan be on a sliding scale depending on the quality of the photographthat the user downloads or the size or quality of the print. Forexample, a photograph may cost $1.00 for 1024×768 resolution or $2.00for 1600×1200 resolution. Similarly, a print may cost $1.00 for 3×5 or$5 for 8×10. For downloads, an “upgrade” may be possible by charging thedifference between the resolutions. An automated process may be used toreduce the number of pixels for purposes of having a lower qualityversion to sell.

In addition, a surcharge may be applied (even if no surcharge isrequired) for various enhancements to the photograph, such as“upconverting” to a higher resolution, eliminating red-eye, enhancingshadow, color, or brightness, etc.

Moreover, when a photographer takes photographs, he can be provided withprinted cards bearing a Uniform Resource Locator (URL) and a unique codein order that the user would be able to enter into the web site to findthe photograph or the series of photographs then being taken. Thephotographer can also distribute cards (the printed cards bearing theURL and the unique code or any other cards known to those skilled in theart) to people whom he photographs, whether intentionally orinadvertently. The photographer can further advertise the same (e.g.,the URL and the unique code) via a mark on his camera, a T-shirt, orother means.

Fixed-place cameras can also serve this function (e.g., the ofphotographer). For example, a camera set up at an intersection inHollywood might take and upload one photograph every 10 seconds.

Photographers can also be given accounts and be allowed to uploadphotographs to the site. The database is populated during this process,although additional database information can be added later by web siteusers. In addition, the number of times the photograph has beenpurchased and/or viewed can be a part of the database.

In one embodiment, the method and apparatus of the present inventionshould be capable of face recognition. It should assign values tovarious factors (i.e., ratio of distance between pupils to distance totip of nose, etc.). It would add this information to the database foruploaded photographs. A user can then upload a photograph of the targetperson and the system would then generate the same data from thatphotograph and use it to limit the possible search targets.

A provider of the present method and apparatus or a photographer canalso hand out pins, clothing, or other materials that are marked in away that allows a computer to later recognize them in a photograph. FIG.2 shows an exemplary distinctive marker 200 having an optical code 202,such as a bar code. The marker may have a color combination, distinctiveshape, lettering, bar code, or other optical pattern, or somecombination of the foregoing, that is unique to the marker. The markermay be computer generated, for example, and produced using an end-user'slaser or ink-jet printer. The marker may be associated with specificinformation, for example, a particular user account, photographer,subject matter type, person, event, or location. Users can later searchfor photographs containing an image of the marker.

Numerous distribution mechanisms exist whereby photographs may bedistributed from a source over a wide area network, such as theInternet. In some cases, the photographs are distributed using acentralized server system (such as Napster 2.0, eBay, or from a website). In other cases, the photographs are distributed using adecentralized system (such as Gnutella). In a preferred implementation,the photographs are distributed to a person using the centralized serversystem or using a central hub.

Embodiments of the present invention operate in accordance with at leastone web-hosting mechanism and a plurality of user mechanismscommunicating over a wide area network, such as the Internet.Specifically, a web-hosting mechanism includes a database, an interfaceapplication and a server, wherein the server is adapted to communicatewith a plurality of user mechanisms over a wide area network. It shouldbe appreciated that the mechanisms described can include, but are notlimited to, personal computers, mainframe computers, personal digitalassistances, wireless communication devices and all other physical andwireless connected network devices generally known to those skilled inthe art. It should further be understood that the database depicted caninclude, but is not limited, to RAM, cache memory, flash memory,magnetic disks, optical disks, removable disks, SCSI disks, IDE harddrives, tape drives, and all other types of data storage devices (andcombinations thereof, such as RAID devices) generally known to thoseskilled in the art. In addition, the mechanisms described above are forpurposes of example only and the invention is not limited thereby.

Having thus described several embodiments for photograph finding, itshould be apparent to those skilled in the art that certain advantagesof the system have been achieved. It should also be appreciated thatvarious modifications, adaptations, and alternative embodiments thereofmay be made within the scope and spirit of the present invention. Forexample, in the context of the present invention a photograph caninclude video, audio, and/or other representation of how people conceiveof the world. The invention is defined by the following claims.

1. A method implemented by a computer system including computerhardware, the method comprising: receiving, at a web site, a firstdigital image selected by a user of the web site; receiving, from afacial recognition process, first information that represents at leastone facial feature appearing within the first digital image; comparingthe first information to at least second information that represents atleast a second facial feature appearing within at least a second digitalimage; based on the comparing, identifying at least the second digitalimage from among a plurality of digital images; and causing presentationof at least the second digital image to the user via the web site,wherein the presentation identifies at least one target person havingfacial features that at least partially match the first information. 2.The method of claim 1, further comprising: receiving, from a facialrecognition process, the second information.
 3. The method of claim 1,further comprising: performing the facial recognition process.
 4. Themethod of claim 1, further comprising: analyzing the first digital imageto generate the first information.
 5. The method of claim 1, furthercomprising: analyzing the second digital image to generate the secondinformation.
 6. The method of claim 1, wherein the second digital imageis submitted by another user of the web site.
 7. The method of claim 1,wherein causing presentation of at least the second digital image to theuser via the web site includes causing the web site to present at leastthe second digital image to the user via a personal computer.
 8. Themethod of claim 1, wherein causing presentation of at least the seconddigital image to the user via the web site includes causing the web siteto present at least the second digital image to the user via a wirelessconnected network device.
 9. The method of claim 1, wherein comparingthe first information includes comparing the first information to secondinformation that is generated from the second digital image before thefirst information is received from the facial recognition process.
 10. Amethod implemented by a computer system including computer hardware, themethod comprising: storing a first digital image uploaded via a webbrowser to a web site, wherein the first digital image contains a visualrepresentation of at least a first human facial feature; receiving, froma facial recognition process, first information that represents thefirst human facial feature; storing a second digital image that containsa visual representation of at least a second human facial feature;receiving, from the facial recognition process, second information thatrepresents the second human facial feature; comparing the firstinformation to at least the second information; based on the comparing,identifying at least the second digital image from among a plurality ofdigital images; and causing presentation of at least the second digitalimage via the web site, wherein the presentation identifies at least onetarget person having facial features that at least partially match thefirst information.
 11. The method of claim 10, further comprising:performing the facial recognition process.
 12. The method of claim 10,further comprising: analyzing the first digital image to generate thefirst information.
 13. The method of claim 10, further comprising:analyzing the second digital image to generate the second information.14. The method of claim 10, wherein causing presentation of at least thesecond digital image via the web site includes causing a presentation ofa web page that identifies the at least one target person.
 15. Themethod of claim 10, wherein causing presentation of at least the seconddigital image via the web site includes causing a presentation of auniversal resource locator (URL) indicative of the second digital image.16. A method, comprising: a computer system receiving a first imageselected by a user of a web site, wherein the first image depicts atleast a first face; the computer system comparing facial recognitioninformation for the first face with facial recognition information forfaces in a plurality of images, wherein the plurality of images includesa second image depicting at least a second face; based on the comparing,the computer system determining that a match exists between the firstface and the second face.
 17. The method of claim 16, furthercomprising: in response to the determining, the computer systemproviding an indication of the match to the user via the web site. 18.The method of claim 17, wherein the indication includes presentation ofa web page that includes the second image.
 19. The method of claim 17,wherein the indication includes presentation of a thumbnail version ofthe second image.
 20. The method of claim 16, further comprising: thecomputer system performing a facial recognition process that extractsthe facial recognition information for the first face from the firstimage.
 21. The method of claim 16, wherein the facial recognitioninformation for the first face includes information indicative ofdistances between facial features.
 22. The method of claim 16, furthercomprising: the computer system storing location information identifyingwhere the second image was taken.
 23. The method of claim 16, furthercomprising: the computer system storing a value indicative of a numberof times that the second image has been viewed.
 24. A method,comprising: a computer system receiving an image uploaded via a webbrowser to a web site; and in response to receiving the uploaded image,the computer system searching a data store to determine whether a facedepicted in the uploaded image is present within one or more otherimages stored in the data store.
 25. The method of claim 24, wherein thesearching includes the computer system comparing facial recognitioninformation for the uploaded image to facial recognition information forother images present in the data store.
 26. The method of claim 24,further comprising: in response to the searching, determining that amatch exists between the face depicted in the uploaded image and a facedepicted in at least one other image in the data store, the computersystem providing an electronic page with information indicative of theat least one other image.
 27. The method of claim 26, wherein theinformation indicative of the at least one other image is a thumbnailimage.
 28. The method of claim 25, further comprising: the computersystem storing the uploaded image in the data store along with thefacial recognition information for the uploaded image.
 29. The method ofclaim 25, further comprising: the computer system performing a facialrecognition process to determine the facial recognition information forthe uploaded image.